

<!DOCTYPE html>
<html class="writer-html5" lang="en" >
<head>
  <meta charset="utf-8" />
  
  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
  
  <title>Overview &mdash; Ceph Documentation</title>
  

  
  <link rel="stylesheet" href="../../_static/ceph.css" type="text/css" />
  <link rel="stylesheet" href="../../_static/pygments.css" type="text/css" />
  <link rel="stylesheet" href="../../_static/graphviz.css" type="text/css" />
  <link rel="stylesheet" href="../../_static/css/custom.css" type="text/css" />

  
  
    <link rel="shortcut icon" href="../../_static/favicon.ico"/>
  

  
  

  

  
  <!--[if lt IE 9]>
    <script src="../../_static/js/html5shiv.min.js"></script>
  <![endif]-->
  
    
      <script type="text/javascript" id="documentation_options" data-url_root="../../" src="../../_static/documentation_options.js"></script>
        <script src="../../_static/jquery.js"></script>
        <script src="../../_static/underscore.js"></script>
        <script src="../../_static/doctools.js"></script>
    
    <script type="text/javascript" src="../../_static/js/theme.js"></script>

    
    <link rel="index" title="Index" href="../../genindex/" />
    <link rel="search" title="Search" href="../../search/" />
    <link rel="next" title="systemd" href="../systemd/" />
    <link rel="prev" title="ceph-volume" href="../" /> 
</head>

<body class="wy-body-for-nav">

   
  <header class="top-bar">
    

















<div role="navigation" aria-label="breadcrumbs navigation">

  <ul class="wy-breadcrumbs">
    
      <li><a href="../../" class="icon icon-home"></a> &raquo;</li>
        
          <li><a href="../">ceph-volume</a> &raquo;</li>
        
      <li>Overview</li>
    
    
      <li class="wy-breadcrumbs-aside">
        
          
            <a href="../../_sources/ceph-volume/intro.rst.txt" rel="nofollow"> View page source</a>
          
        
      </li>
    
  </ul>

  
  <hr/>
</div>
  </header>
  <div class="wy-grid-for-nav">
    
    <nav data-toggle="wy-nav-shift" class="wy-nav-side">
      <div class="wy-side-scroll">
        <div class="wy-side-nav-search"  style="background: #eee" >
          

          
            <a href="../../">
          

          
            
            <img src="../../_static/logo.png" class="logo" alt="Logo"/>
          
          </a>

          

          
<div role="search">
  <form id="rtd-search-form" class="wy-form" action="../../search/" method="get">
    <input type="text" name="q" placeholder="Search docs" />
    <input type="hidden" name="check_keywords" value="yes" />
    <input type="hidden" name="area" value="default" />
  </form>
</div>

          
        </div>

        
        <div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">
          
            
            
              
            
            
              <ul class="current">
<li class="toctree-l1"><a class="reference internal" href="../../start/intro/">Ceph 简介</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../install/">安装 Ceph</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../cephadm/">Cephadm</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../rados/">Ceph 存储集群</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../cephfs/">Ceph 文件系统</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../rbd/">Ceph 块设备</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../radosgw/">Ceph 对象网关</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../mgr/">Ceph 管理器守护进程</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../mgr/dashboard/">Ceph 仪表盘</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../api/">API 文档</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../architecture/">体系结构</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../dev/developer_guide/">开发者指南</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../dev/internals/">Ceph 内幕</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../governance/">项目管理</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../foundation/">Ceph 基金会</a></li>
<li class="toctree-l1 current"><a class="reference internal" href="../">ceph-volume</a><ul class="current">
<li class="toctree-l2 current"><a class="reference internal" href="../#id2">迁移</a><ul class="current">
<li class="toctree-l3"><a class="reference internal" href="../#id3">全新部署</a></li>
<li class="toctree-l3 current"><a class="reference internal" href="../#osd">已有 OSD 怎么办</a><ul class="current">
<li class="toctree-l4 current"><a class="current reference internal" href="#">Overview</a></li>
<li class="toctree-l4"><a class="reference internal" href="#replacing-ceph-disk">Replacing <code class="docutils literal notranslate"><span class="pre">ceph-disk</span></code></a></li>
<li class="toctree-l4"><a class="reference internal" href="#gpt-partitions-are-simple">GPT partitions are simple?</a></li>
<li class="toctree-l4"><a class="reference internal" href="#modularity">Modularity</a></li>
<li class="toctree-l4"><a class="reference internal" href="#ceph-volume-lvm"><code class="docutils literal notranslate"><span class="pre">ceph-volume</span> <span class="pre">lvm</span></code></a></li>
<li class="toctree-l4"><a class="reference internal" href="#lvm-performance-penalty">LVM performance penalty</a></li>
<li class="toctree-l4"><a class="reference internal" href="../systemd/">systemd</a></li>
<li class="toctree-l4"><a class="reference internal" href="../inventory/"><code class="docutils literal notranslate"><span class="pre">inventory</span></code></a></li>
<li class="toctree-l4"><a class="reference internal" href="../drive-group/"><code class="docutils literal notranslate"><span class="pre">drive-group</span></code></a></li>
<li class="toctree-l4"><a class="reference internal" href="../lvm/"><code class="docutils literal notranslate"><span class="pre">lvm</span></code></a></li>
<li class="toctree-l4"><a class="reference internal" href="../lvm/activate/"><code class="docutils literal notranslate"><span class="pre">activate</span></code></a></li>
<li class="toctree-l4"><a class="reference internal" href="../lvm/batch/"><code class="docutils literal notranslate"><span class="pre">batch</span></code></a></li>
<li class="toctree-l4"><a class="reference internal" href="../lvm/batch/#ceph-volume-lvm-batch-report">报表</a></li>
<li class="toctree-l4"><a class="reference internal" href="../lvm/batch/#sizing">Sizing</a></li>
<li class="toctree-l4"><a class="reference internal" href="../lvm/batch/#idempotency-and-disk-replacements">Idempotency and disk replacements</a></li>
<li class="toctree-l4"><a class="reference internal" href="../lvm/encryption/">加密</a></li>
<li class="toctree-l4"><a class="reference internal" href="../lvm/prepare/"><code class="docutils literal notranslate"><span class="pre">prepare</span></code></a></li>
<li class="toctree-l4"><a class="reference internal" href="../lvm/create/"><code class="docutils literal notranslate"><span class="pre">create</span></code></a></li>
<li class="toctree-l4"><a class="reference internal" href="../lvm/scan/">scan</a></li>
<li class="toctree-l4"><a class="reference internal" href="../lvm/systemd/">systemd</a></li>
<li class="toctree-l4"><a class="reference internal" href="../lvm/list/"><code class="docutils literal notranslate"><span class="pre">list</span></code></a></li>
<li class="toctree-l4"><a class="reference internal" href="../lvm/zap/"><code class="docutils literal notranslate"><span class="pre">zap</span></code></a></li>
<li class="toctree-l4"><a class="reference internal" href="../lvm/migrate/"><code class="docutils literal notranslate"><span class="pre">migrate</span></code></a></li>
<li class="toctree-l4"><a class="reference internal" href="../lvm/newdb/"><code class="docutils literal notranslate"><span class="pre">new-db</span></code></a></li>
<li class="toctree-l4"><a class="reference internal" href="../lvm/newwal/"><code class="docutils literal notranslate"><span class="pre">new-wal</span></code></a></li>
<li class="toctree-l4"><a class="reference internal" href="../simple/"><code class="docutils literal notranslate"><span class="pre">simple</span></code></a></li>
<li class="toctree-l4"><a class="reference internal" href="../simple/activate/"><code class="docutils literal notranslate"><span class="pre">activate</span></code></a></li>
<li class="toctree-l4"><a class="reference internal" href="../simple/scan/"><code class="docutils literal notranslate"><span class="pre">scan</span></code></a></li>
<li class="toctree-l4"><a class="reference internal" href="../simple/systemd/">systemd</a></li>
<li class="toctree-l4"><a class="reference internal" href="../zfs/"><code class="docutils literal notranslate"><span class="pre">zfs</span></code></a></li>
<li class="toctree-l4"><a class="reference internal" href="../zfs/inventory/"><code class="docutils literal notranslate"><span class="pre">inventory</span></code></a></li>
</ul>
</li>
</ul>
</li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../../releases/general/">Ceph 版本（总目录）</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../releases/">Ceph 版本（索引）</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../security/">Security</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../glossary/">Ceph 术语</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../jaegertracing/">Tracing</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../translation_cn/">中文版翻译资源</a></li>
</ul>

            
          
        </div>
        
      </div>
    </nav>

    <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">

      
      <nav class="wy-nav-top" aria-label="top navigation">
        
          <i data-toggle="wy-nav-top" class="fa fa-bars"></i>
          <a href="../../">Ceph</a>
        
      </nav>


      <div class="wy-nav-content">
        
        <div class="rst-content">
        
          <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
           <div itemprop="articleBody">
            
<div id="dev-warning" class="admonition note">
  <p class="first admonition-title">Notice</p>
  <p class="last">This document is for a development version of Ceph.</p>
</div>
  <div id="docubetter" align="right" style="padding: 5px; font-weight: bold;">
    <a href="https://pad.ceph.com/p/Report_Documentation_Bugs">Report a Documentation Bug</a>
  </div>

  
  <div class="section" id="overview">
<span id="ceph-volume-overview"></span><h1>Overview<a class="headerlink" href="#overview" title="Permalink to this headline">¶</a></h1>
<p>The <code class="docutils literal notranslate"><span class="pre">ceph-volume</span></code> tool aims to be a single purpose command line tool to deploy
logical volumes as OSDs, trying to maintain a similar API to <code class="docutils literal notranslate"><span class="pre">ceph-disk</span></code> when
preparing, activating, and creating OSDs.</p>
<p>It deviates from <code class="docutils literal notranslate"><span class="pre">ceph-disk</span></code> by not interacting or relying on the udev rules
that come installed for Ceph. These rules allow automatic detection of
previously setup devices that are in turn fed into <code class="docutils literal notranslate"><span class="pre">ceph-disk</span></code> to activate
them.</p>
</div>
<div class="section" id="replacing-ceph-disk">
<span id="ceph-disk-replaced"></span><h1>Replacing <code class="docutils literal notranslate"><span class="pre">ceph-disk</span></code><a class="headerlink" href="#replacing-ceph-disk" title="Permalink to this headline">¶</a></h1>
<p>The <code class="docutils literal notranslate"><span class="pre">ceph-disk</span></code> tool was created at a time were the project was required to
support many different types of init systems (upstart, sysvinit, etc…) while
being able to discover devices. This caused the tool to concentrate initially
(and exclusively afterwards) on GPT partitions. Specifically on GPT GUIDs,
which were used to label devices in a unique way to answer questions like:</p>
<ul class="simple">
<li><p>is this device a Journal?</p></li>
<li><p>an encrypted data partition?</p></li>
<li><p>was the device left partially prepared?</p></li>
</ul>
<p>To solve these, it used <code class="docutils literal notranslate"><span class="pre">UDEV</span></code> rules to match the GUIDs, that would call
<code class="docutils literal notranslate"><span class="pre">ceph-disk</span></code>, and end up in a back and forth between the <code class="docutils literal notranslate"><span class="pre">ceph-disk</span></code> systemd
unit and the <code class="docutils literal notranslate"><span class="pre">ceph-disk</span></code> executable. The process was very unreliable and time
consuming (a timeout of close to three hours <strong>per OSD</strong> had to be put in
place), and would cause OSDs to not come up at all during the boot process of
a node.</p>
<p>It was hard to debug, or even replicate these problems given the asynchronous
behavior of <code class="docutils literal notranslate"><span class="pre">UDEV</span></code>.</p>
<p>Since the world-view of <code class="docutils literal notranslate"><span class="pre">ceph-disk</span></code> had to be GPT partitions exclusively, it meant
that it couldn’t work with other technologies like LVM, or similar device
mapper devices. It was ultimately decided to create something modular, starting
with LVM support, and the ability to expand on other technologies as needed.</p>
</div>
<div class="section" id="gpt-partitions-are-simple">
<h1>GPT partitions are simple?<a class="headerlink" href="#gpt-partitions-are-simple" title="Permalink to this headline">¶</a></h1>
<p>Although partitions in general are simple to reason about, <code class="docutils literal notranslate"><span class="pre">ceph-disk</span></code>
partitions were not simple by any means. It required a tremendous amount of
special flags in order to get them to work correctly with the device discovery
workflow. Here is an example call to create a data partition:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="o">/</span><span class="n">sbin</span><span class="o">/</span><span class="n">sgdisk</span> <span class="o">--</span><span class="n">largest</span><span class="o">-</span><span class="n">new</span><span class="o">=</span><span class="mi">1</span> <span class="o">--</span><span class="n">change</span><span class="o">-</span><span class="n">name</span><span class="o">=</span><span class="mi">1</span><span class="p">:</span><span class="n">ceph</span> <span class="n">data</span> <span class="o">--</span><span class="n">partition</span><span class="o">-</span><span class="n">guid</span><span class="o">=</span><span class="mi">1</span><span class="p">:</span><span class="n">f0fc39fd</span><span class="o">-</span><span class="n">eeb2</span><span class="o">-</span><span class="mi">49</span><span class="n">f1</span><span class="o">-</span><span class="n">b922</span><span class="o">-</span><span class="n">a11939cf8a0f</span> <span class="o">--</span><span class="n">typecode</span><span class="o">=</span><span class="mi">1</span><span class="p">:</span><span class="mi">89</span><span class="n">c57f98</span><span class="o">-</span><span class="mi">2</span><span class="n">fe5</span><span class="o">-</span><span class="mi">4</span><span class="n">dc0</span><span class="o">-</span><span class="mi">89</span><span class="n">c1</span><span class="o">-</span><span class="n">f3ad0ceff2be</span> <span class="o">--</span><span class="n">mbrtogpt</span> <span class="o">--</span> <span class="o">/</span><span class="n">dev</span><span class="o">/</span><span class="n">sdb</span>
</pre></div>
</div>
<p>Not only creating these was hard, but these partitions required devices to be
exclusively owned by Ceph. For example, in some cases a special partition would
be created when devices were encrypted, which would contain unencrypted keys.
This was <code class="docutils literal notranslate"><span class="pre">ceph-disk</span></code> domain knowledge, which would not translate to a “GPT
partitions are simple” understanding. Here is an example of that special
partition being created:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="o">/</span><span class="n">sbin</span><span class="o">/</span><span class="n">sgdisk</span> <span class="o">--</span><span class="n">new</span><span class="o">=</span><span class="mi">5</span><span class="p">:</span><span class="mi">0</span><span class="p">:</span><span class="o">+</span><span class="mi">10</span><span class="n">M</span> <span class="o">--</span><span class="n">change</span><span class="o">-</span><span class="n">name</span><span class="o">=</span><span class="mi">5</span><span class="p">:</span><span class="n">ceph</span> <span class="n">lockbox</span> <span class="o">--</span><span class="n">partition</span><span class="o">-</span><span class="n">guid</span><span class="o">=</span><span class="mi">5</span><span class="p">:</span><span class="kc">None</span> <span class="o">--</span><span class="n">typecode</span><span class="o">=</span><span class="mi">5</span><span class="p">:</span><span class="n">fb3aabf9</span><span class="o">-</span><span class="n">d25f</span><span class="o">-</span><span class="mi">47</span><span class="n">cc</span><span class="o">-</span><span class="n">bf5e</span><span class="o">-</span><span class="mi">721</span><span class="n">d181642be</span> <span class="o">--</span><span class="n">mbrtogpt</span> <span class="o">--</span> <span class="o">/</span><span class="n">dev</span><span class="o">/</span><span class="n">sdad</span>
</pre></div>
</div>
</div>
<div class="section" id="modularity">
<h1>Modularity<a class="headerlink" href="#modularity" title="Permalink to this headline">¶</a></h1>
<p><code class="docutils literal notranslate"><span class="pre">ceph-volume</span></code> was designed to be a modular tool because we anticipate that
there are going to be lots of ways that people provision the hardware devices
that we need to consider. There are already two: legacy ceph-disk devices that
are still in use and have GPT partitions (handled by <a class="reference internal" href="../simple/#ceph-volume-simple"><span class="std std-ref">simple</span></a>),
and lvm. SPDK devices where we manage NVMe devices directly from userspace are
on the immediate horizon, where LVM won’t work there since the kernel isn’t
involved at all.</p>
</div>
<div class="section" id="ceph-volume-lvm">
<h1><code class="docutils literal notranslate"><span class="pre">ceph-volume</span> <span class="pre">lvm</span></code><a class="headerlink" href="#ceph-volume-lvm" title="Permalink to this headline">¶</a></h1>
<p>By making use of <a class="reference internal" href="../../glossary/#term-LVM-tags"><span class="xref std std-term">LVM tags</span></a>, the <a class="reference internal" href="../lvm/#ceph-volume-lvm"><span class="std std-ref">lvm</span></a> sub-command is
able to store and later re-discover and query devices associated with OSDs so
that they can later be activated.</p>
</div>
<div class="section" id="lvm-performance-penalty">
<h1>LVM performance penalty<a class="headerlink" href="#lvm-performance-penalty" title="Permalink to this headline">¶</a></h1>
<p>In short: we haven’t been able to notice any significant performance penalties
associated with the change to LVM. By being able to work closely with LVM, the
ability to work with other device mapper technologies was a given: there is no
technical difficulty in working with anything that can sit below a Logical Volume.</p>
</div>



           </div>
           
          </div>
          <footer>
    <div class="rst-footer-buttons" role="navigation" aria-label="footer navigation">
        <a href="../systemd/" class="btn btn-neutral float-right" title="systemd" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a>
        <a href="../" class="btn btn-neutral float-left" title="ceph-volume" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a>
    </div>

  <hr/>

  <div role="contentinfo">
    <p>
        &#169; Copyright 2016, Ceph authors and contributors. Licensed under Creative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0).

    </p>
  </div> 

</footer>
        </div>
      </div>

    </section>

  </div>
  

  <script type="text/javascript">
      jQuery(function () {
          SphinxRtdTheme.Navigation.enable(true);
      });
  </script>

  
  
    
   

</body>
</html>